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1. Introduction 

Providing a microscopic expression for the entropy production has been one of the grand 
aims of statistical mechanics, going back to the seminal work of Boltzmann. However, 
both the range of validity of the second law and of its proposed derivations have, from 
the very beginning, generated discussion and controversy. The recent discovery of the 
fluctuation and work theorems [H [21 [3l HI [5l [6] has re-invigorated the debate. These 
intriguing results provide equalities involving the probability distributions for work 
or entropy production, and appear to be of specific interest for the study of small 
systems [3, El E]. With respect to the average value of work or entropy production, 
they are consistent with the second law, but provide no additional information beyond 
it. In a recent work [10] (see also [11]) we have derived, from first principles, the exact 
value of the average dissipated work (ly)diss upon bringing a system from one canonical 
equilibrium state at a temperature T into another one at the same temperature. The 
dissipated work is defined as the extra amount of work, on top of the difference of free 
energy AF, that is required for making this transition. Its expression reads: 

(l^)diss = (W) -AF = kT(\n^y (1) 

Here k is the Boltzmann constant and p = p(r; t) is the probability density in phase 
space to observe the system to be in a micro-state T = (g, p) specified by a set of positions 
q and momenta p at an intermediate time t. The averaging brackets denote an average 
with respect to the density p. The other density p = p{T; t) represents the distribution 
in the time-reversed process observed at a corresponding time-reversed phase point 
r = (g, —p) at the same time as the forward process. As an example, consider 
compression and expansion of a gas in a cylinder, a forward experiment corresponding 
to the expansion of a gas by moving a piston from an initial to a final position; the 
backward experiment corresponds to performing the motion of the piston in the time- 
reversed manner. The statistics of the micro-states are taken at the same intermediate 
position of the piston. Both forward and backward experiment are assumed to start in 
canonical equilibrium at the same temperature. 

The dissipated work ([T]) can also be written in terms of a well known and powerful 
concept from information theory, the relative entropy or Kullback-Leibler distance 
between two probability densities [T2] : 

(W^)diss = kTD{p\\p), (2) 

where 

D{p\\p)= /rfrp(r,t)ln^ (3) 

= J dTp{T, t) In p(r, t)- J dTp{T, t) In p(f, t) . (4) 

is the relative entropy between p and p. Written in this way, the result reveals its 
deep meaning, linking the dissipation directly to the irreversibility of the underlying 
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experiment. Indeed the relative entropy between probability densities is expressing 
the difficulty of distinguishing samplings from these densities [T2]. In the present 
case, it measures the difficulty of distinguishing whether observed data of the micro- 
state correspond to those from a forward or backward experiment. Therefore, the 
relative entropy (jl]) can be considered as a quantitative measure of the arrow of time 



At the end of the forward process, the system is generally speaking in a 
nonequilibrium state, for which the entropy is not well defined. Hence, to make 
the connection with the entropy production, we introduce an idealized heat bath at 
temperature T, with which the system is put in contact at the end of the forward 
experiment, without extra work. The system thus relaxes back to canonical equilibrium, 
which is also the starting state of the backward process. During the relaxation, the 
system transfers an average amount of heat Q to the bath. We can now evaluate the 
total entropy production. On the one hand, the entropy change in the bath, assuming 
it operates quasi-statically, is given by Q/T. On the other hand, the entropy change 
between the canonical equilibrium state of initial and final states in the system reads: 
{AU - AF)/T = {{W) - Q- AF)/T. Here we used the relation F = U -TS, where 
U = (H) is the equilibrium internal energy, calculated with respect to the prevailing 
canonical distribution, and S the thermodynamic entropy of the system. The total 
entropy production (in the total device, system plus heat bath) in the forward process, 
AS, is thus equal to the dissipated (W^)diss divided by temperature, hence: 



This result goes right to the core of the second law, as it provides an exact explicit 
microscopic expression for the total entropy production in the considered scenario. 

This scenario is very specific: the system is initially at canonical equilibrium, and 
disconnected from any heat bath during the perturbation [10]. These assumptions 
greatly simplify the derivation, but also generate the misleading impression that they 
are essential. The aim of this paper is to further clarify the status of the basic results 
(IS]), and to show that it has a wider range of validity. 

Our discussion will be based on a general and exact mathematical identity for 
the relative entropy, derived in Sec. [2], which is valid for arbitrary initial conditions 
in the forward and backward process. In Sec. [3l this identity is applied to different 
scenarios, including various types of initial equilibrium states, such as grand canonical 
ensembles, constrained equilibrium states, and multi-canonical distributions at different 
temperatures. In each of these cases, the entropy production is (aside a factor k) equal 
to the relative entropy of the phase space density between the forward and the backward 
states, provided some idealized relaxation process is assumed at the end of the forward 
process. In Sec. H] we present the quantum analogue of the above result, expressed in 
terms of the quantum relative entropy [261 EH EH] . 



[laiaiiaiisiiniiiHiiTgiisoiiaiEaEsiiMiEg 
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2. General Formulation 

We consider a system described by tlie Hamiltonian H{r] A), with A a control parameter 
that describes the energy exchange with an external device. For simphcity, we will 
assume that the Hamiltonian is an even function with respect to inversion of momenta. 
As is well known, the probability density in phase space p(T, t) to observe the system 
in a specific micro-state F at time t obeys the Liouville equation: 

^ = Mr.«). (6) 

In a classical system, the Liouville operator £ is given by Poisson brackets: Cp = 
{H, p} = dqHdpP — dpHdqP (summation convention over all positions and momenta is 
assumed) . 

An important implication of the Liouville theorem is that the Shannon entropy 
associated to p, namely: 

■rfFp(F)lnp(F) (7) 

is constant in time, as can be easily proven by applying the Liouville equation followed 
by partial integration This appears to be in contradiction with the second law 

of thermodynamics, or at least precludes the use of this expression as a microscopic 
definition for the entropy. In fact, we share the general opinion that entropy is 
not defined at the microscopic level except when the system is at equilibrium. The 
Shannon entropy reduces to the proper thermodynamic entropy only for systems at 
equilibrium. 

An essential remark for the further discussion is that the above property remains 
valid for a smooth time-dependent Hamiltonian or Liouvillian. We will henceforth focus 
on scenarios where the control parameter A is changed from an initial value A^ to a 
final value A^, according to a specific protocol X{t),t G [0,r]. We also consider a 
time-reversed experiment, starting at the final value A^ of the control parameter, and 
performing the time-reversed perturbation. The quantities appearing in this setting will 
be indicated by a superscript tilde. We will always use the forward time t to designate 
time in both forward and backward experiment, keeping in mind that the corresponding 
real time in the backward experiment is equal to r — t. The initial conditions for both 
forward and backward experiments are for the moment left unspecified. The above set- 
up is reminiscent of the one introduced by Jarzynski in his derivation of the Jarzynski 
equality |4], although the scope of our derivation extends, as we will see, well beyond 
the validity of the Jarzynski equality. 

The key observation is that the phase space density of backward evolution with 
reversed momenta, namely p(T,t), obeys the same Liouville equation ([6]) with respect 
to the forward time variable t: 

'^ = Cp(r,t). (8) 

The transition from Eq. ([6]) to Eq. ([8]) is based on micro-reversibility: trajectories 
retrace their steps upon reversing the schedule of A and inverting momenta. 
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Mathematically, the result follows easily by writing the Liouville equation for p in its 
proper time t — t. When switching to the forward time t, the negative sign, due to the 
time derivative in the l.h.s. of the Liouville equation, is canceled by another one in the 
r.h.s., appearing upon inverting the momenta. As a particular example, for the case of a 
time-independent Hamiltonian, the above statement reduces to the fact that p{T, t) and 
p(r, T — t), obey the same Liouville equation, when the time-derivative is with respect 
to t. 

We can now invoke a remarkable property for probability distributions that obey 
the same Liouville equation: their relative entropy is invariant in time [161 129] • In 
particular, the following quantity: 

I rfrp(r,t)inp(f,t) (9) 

is time-invariant, as can again be easily verified by applying Liouville's equation followed 
by partial integration. 

The invariance in time of ([7]) and (Q now allows us to rewrite the quantity of central 
interest, namely the relative entropy (jlj), as follows. Since each term in the r.h.s. of 
Eq. (jl]) is invariant under time translation, we can shift time in the first term back to 
t = 0, and in the second term forward to t = r: 

D[p{T,t)\\p{t,t)) = I drp(r,0)lnp(r,0)- J cirp(r,r)lnp(f,r). (10) 

We stress again that this is an exact result, valid for any initial conditions for p and p. 
For specific choices of these states, we proceed to make the connection with the entropy 
production in a variety of scenarios. Note also that the above result will only be usefuU 
if the support of p includes the support of p, i.e., there are no phase points for which 
p = and p 7^ 0. Otherwise D{p\\p) diverges. 



3. Explicit entropy production in different scenarios 

Along this section, we apply Eq. ( fTOl) for specific choices of equilibrium initial conditions 
p(r, 0) and p(r, r) in forward and backward process, respectively. We will see that in all 
the considered cases, the resulting relative entropy is equal to the entropy production 
along the forward process. If p(r, 0) is an equilibrium distribution or the product of 
independent equilibrium distributions, Shannon entropy is equal to the thermodynamic 
entropy: 

S{0) = -k J dTp{T, 0) lnp(r, 0). (11) 

This equality holds for different equilibrium ensembles, such as canonical and grand 
canonical. With this choice of initial condition for the forward process the first term in 
Eq. (|TOl) is minus the initial equilibrium entropy. In the following, we will prove that 
the second term can be identified with the final entropy. 

As mentioned in previous sections, in this paper we only consider as meaningful 
the entropy of equilibrium states. On the other hand, after the forward process is 
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completed at time r, the resulting state p(r, r) is not at equilibrium. In order to 
have a well defined expression for the entropy production, we need some relaxation 
mechanism to equilibrium after the forward process has been completed. The function 
of this mechanism is twofold: it allows us to have a meaningful definition of entropy 
production, and also drives the system from the non equilibrium final state p(r,r_) to 
the equilibrium one p(r, r), which is the initial condition for the backward process|il 

Notice that the relaxation from p(r, r) to p(r, r) in an isolated system is 
incompatible with the Liouville theorem. Our derivation does not touch on this old and 
unresolved problem. Instead, for the relaxation to p(r, r), we invoke the presence of one 
or several ideal (super)baths, to which the system is weakly coupled. This relaxation can, 
in principle, be performed without any extra energy input, since coupling the superbath 
with just one degree of freedom of the system is enough to induce thermalization 
|30] . Moreover, if this degree of freedom is chosen to have zero mean (for instance, 
one of the momenta of the bath), then the average work done by switching on the 
coupling is strictly zero. Finally, we will assume that the entropy production in the 
superbaths, due to exchange of energy and/or particles with the system, is given by the 
standard expressions from statistical mechanics and thermodynamics. Note that such or 
similar assumptions also appear in the various derivations of both fiuctuation and work 
theorems, and in the study of nonequilibrium steady states. They are also intrinsically 
present in the formulations via mesoscopic stochastic descriptions. 



3.1. Canonical distributions 

In our first example, we start the forward process with a canonical distribution at 
temperature T and, after the process is completed at time r, we connect the system 
with a thermal bath at a different temperature T'. The initial condition for the backward 
process is in this canonical distribution at temperature T': 

-l3H(r;\A) 

Z(T, A) = / dTe~'^^^^'^^ being the partition function. Recalling that we consider 
Hamiltonians even functions of the momenta, we obtain from Eq. flTUl) and flTT]) : 

kDMlp) = -S(0) + ^-^^^^^^2:^ (14) 

where averages {■)t are taken over the forward time density p(r, r) and 

F{T,X) = -kT\nZ{T,X). (15) 

J This relaxation will take some time Tioi. However, recall that our time variable t in p{T,t) denotes 
the stage of the process, using the forward one as reference, rather than the real physical time tphys- In 
a complete forward-backward cycle, the time variable in p(r, t) will he t = 2t + Trei ^ iphys, so p{T, t) 
is the state of the system at iphys = t + Tj-d). 
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is the usual equilibrium free energy. After relaxation to equilibrium with the bath at 
temperature T', the average energy of the system becomes (if)eq^^, which is only a 
function of and T'. In the relaxation, the system transfers an amount of energy to 
the thermal bath Q = {H)j. — {H)cq,r- Therefore: 

kD(p\\p) = - s(o) + («U.r-Hr,XB) ^ Q 

= - S(0) + S(t) + ASt^h = AS (16) 

where S{t) is the final entropy of the system (in equilibrium at temperature T'), and 
^'S'bath is the increase of the entropy in the bath. 

Therefore, the relative entropy is equal to the total entropy production along the 
forward process plus the final relaxation: AS = kD{p\\p). Notice that all entropies 
have been calculated only for equilibrium states. In fact, this example illustrates that 
it makes no sense to talk about the entropy produced along the process if one does not 
specify the final equilibrium state and how is reached. In our case, the total entropy 
production depends on the final temperature T' . Hence, the same forward process \{t) 
can have different entropy production depending on the final temperature T' . 



3.2. General equilibrium distribution 

The above example can be easily generalized to an arbitrary equilibrium distribution 
defined over some conserved quantities Ai{r) {i = 1, . . . ,r), with conjugated variables 
tti = TdS/dAi and the corresponding thermodynamic potential U(T,d) = {H) + 
J2iCii{Ai) — TS. After the forward process, the system is coupled to a reservoir 
characterized by temperature T' and certain values of the conjugated variables. The 
initial condition for the backward process reads: 



p(r, r) = exp 

Hence 



-/?' if(r,AB) + E":^^(r)-w(T', 



a 



i=l 



:i7) 



During relaxation, there is a transfer of energy and of the conserved quantities A, 
from the system to the reservoir given, respectively, hj Q = {H)^ — (if)eq,r and 
AAj = (Aj)^ — (Aj)eq^^. The final entropy of the system is 

^ {H).^,r + ELl «^(A(r))eq,. - U{T\ ^^^^ 

and the entropy increase in the reservoir is AS'bath = {Q + Y.iOi'i^Ai) jT' . The cross 
term ( fTSl) is then equal to S{t) + AS'bath- If, along the forward process, the external 
agent moving \{t) does not undergo any change of entropy, we again find Eq. ([5]): 
^S = kD{p\\p). 

Notice that the cross entropy ( fTSi) contains precisely the non equilibrium average 
values, {H)t and (Aj(r))^, yielding the correct expression for the final equilibrium 
entropy of the system S{t) plus the entropy increase in the bath AS'bath- 
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As a particular case, consider A = N the number of particles. Then, —a = /i, the 
chemical potential. In equilibrium, the thermodynamic potential 

W(T, fi) = (/7)eq - /i(iV)eq - TS (20) 

is the so-called grand canonical potential, where the averages (■)eq are taken with respect 
to the grand canonical ensemble 

p(r,iV) = e-'^[^(r'")-'^^-"(^'^)] (21) 

We now choose as initial condition for the forward process a grand canonical ensemble 
characterized by temperature T and chemical potential /i. We proceed with the forward 
process by changing A(t) (A^ remaining constant to avoid change of entropy in the 
external agent moving A). After the process is completed we connect the system with 
a particle and energy reservoir at temperature T' and chemical potential fi'. In the 
relaxation, there will be an exchange of energy and particles between the system and the 
bath, as explained above. Finally, the total entropy production along the forward process 
plus the relaxation is given by the relative entropy between forward and backward states, 
Eq. dSD. 



3.3. Multicanonical distribution 

The above examples can be extended to the product of several equilibrium states. 
We consider the special case where systems at different temperatures are brought into 
contact with each other. This case is of particular interest since thermal exchange is 
another standard example of entropy production. Also, the present scenario produces 
in the appropriate limit, e.g., the limit of infinitely large thermal reservoirs, a stationary 
nonequilibrium state. We will thus show that our formula of entropy production as a 
relative phase space entropy also apphes in this example of a nonequilibrium steady 
state. 

Consider n systems i = 1, ■ ■ ■ ,n, at the initial time decoupled, and each one at 
canonical equilibrium at temperature Tf. 

i=l ^il-' j) -^A) 

We will assume a similar initial state for the backward process, decoupled systems but 
with arbitrary temperatures i.e.: 

P(r,r) = n y(r' X) - (23) 

i=l ^i\J-i','^B) 

While the systems are decoupled both at initial and final times, thermal contact with 
each other can be established during intermediate stages of the process by an appropriate 
choice of the time-dependent Hamiltonian. One possibility is adding to the original 
decoupled Hamiltonians a coupling term inducing energy exchange and multiplied by 
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the control parameter A(t), set equal to zero at initial and final times. Eq. fllUp gives us 
the following result: 

(^^.(Aa))o 



kD{p\\p) = E 

i=l 

+ E 



1=1 



Ti 
T' 



- k\nZi{Ti, Xa 
+ k\nZ,{r,XB) 



(24) 



As in our previous examples, each subsystem transfers to their corresponding thermal 
baths an energy Qi = (//^(Ab))^ - {Hi{XB))eq,T, {Hi{XB))eq,T being the equilibrium 
energy of each subsystem after the relaxation to p(r,r). Therefore: 



kD{p\\p) = E 



i=l L 



{H,{Xa))o - F{ Ti,XA) ^ {H,{XB))e q,r - F{Ti Xb) + Qi 



T' 



J2 [-S^{0) + Siir) + ASbath,] = AS 



(25) 



i=l 



Hence, the total entropy produced in the entire forward process (plus relaxation) is again 
given by the relative entropy, Eq. Similar arguments can be applied to multi-grand 
canonical distributions, allowing chemical reactions and exchange of particles between 
the system and the final reservoirs. 



3.4- Constrained canonical distributions and filters 

Finally, we now apply the general result Eq. ffTOl) to the case of canonical states restricted 
to subdomains of the phase space. One can imagine two scenarios leading to such 
restricted initial states, namely: (a) the system is subject to constraints in the initial 
and/or final states, or (b) one selects (or restricts the observation to) that specific set 
of trajectories that lie within a prescribed subdomain of phase space; this selection can, 
in principle, be performed at any intermediate time of the process. 

In both cases, the relevant initial condition for the forward process can be written 
as follows: 

where the characteristic function is defined as 

' 1 T eD 
otherwise 



Xd{T) 



(27) 



for any subset D in the phase space. The partition function is then defined as 
Z{(3,X-D)=J^dre-^^(^''\ 

As initial condition for the backward process we consider: 

-/3H(r;Afl) 

The prescriptions for Da and Db are as follows. For case (a), the subsets Da and 
Db reproduce the imposed initial constraints for the forward and backward processes. 
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respectively. For case (b), the set of filtered forward trajectories equals at the initial 
time t = and at the final time r. For simplicity, we will assume that -D^ is 
invariant under inversion of momenta p —p. 
The relative entropy (fTOj) now reads: 

Z{/3, Xa;Da) 



kTD{p\\p) = {H{T; Ab)). - {H{T; Aa))o - kTln 



Z{(3, Xb;Db) 



kTD{p\\p) = {W) - AF - kTln 



(30) 



There are two possible interpretations of this result. If the restriction of microstates to 
Da and Db is due to a filter, as in scenario (b), the initial and final free energies must 
be calculated over the whole phase space. In this case, we have 

^ Zi(3,XA;DA)Zi(3, XbY 
Z{P,Xa)Z{I3,Xb]Db) 

The ratios between the restricted and the non-restricted partition functions are precisely 
the probability that an arbitrary trajectory passes the filter in the forward {pda) 
and backward processes {pdb)-> respectively. Therefore, we obtain the following exact 
equality: 

kTDip\\p) + kThi^ = iW)-AF. (31) 

VDb 

In particular, the non-negativeness of the relative entropy yields the following inequality: 
(m - AF > fcTln^ (32) 



which was derived in [TO] using the Liouville theorem and applied to a Brownian version 
of the Szilard engine and the restore-to-zero process introduced by Landauer. 

On the other hand, if the restriction is due to constraints in the system at t = 
and r, the logarithm of the restricted partition functions can be considered as the actual 
initial and final free energies. Hence, we recover the same result as in the canonical case: 

kTD{p\\p) = {W) -AF= (ly)diss (33) 

It is worthwhile to illustrate the last case on a concrete example, especially since the 
Jarzynksi and Crooks equalities are failing for this kind of scenario [31]. We consider 
the case of free expansion of a gas from a volume V/2 to a final volume V. The 
Hamiltonian in this case is time- independent, but we can still apply our theory to this 
process consisting of a free relaxation from a restricted initial condition. In the forward 
process, the gas is initially in canonical equilibrium, but confined to one half of the 
container, with vacuum in the other half. It is subsequently allowed to expand and fill 
the entire container. Since the Hamiltonian is time independent, the backward process 
starts with the gas at equilibrium filling the whole volume V of the container, and 
obviously remaining so for all times. We thus have 

p(r,r) = ^^xWr) 
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where the characteristic function Xv(r) is 1 for microstates with all the particles in the 
volume V and otherwise. Now the relative entropy is 

kTD{p\\p) = {H)^ - {H)o - kT\n^^^ = -AF (34) 

since the energy is conserved along the evolution {H has no explicit time-dependence) . 
There is no work performed in this experiment, (W) = 0, and thus — AF is the 
dissipative work (the work that we waste compared to a reversible expansion). Therefore, 
Eqs. ([1]) and ([5]) remain valid, even though the Jarzynski and Crooks relations no longer 
hold. 

4. Quantum version 

Just as the Jarzynski equality has been shown to remain valid for quantum processes [321 
[33llH[35llMl[371|3g[39lllQlllIlll2], we now show that our approach can easily be 
extended to the quantum. Time evolution in quantum mechanics is ruled by unitary 
transformations. In particular the density operator obeys the following quantum version 
of the Liouville equation: 

^^p = Cp = -'-[H,p] (35) 

Under this unitary evolution, the Von Neumann entropy S^{p) = — A;Tr(plnp) is 
preserved. For the time-reversed process, the time-reversed density operator p = 0pG~^ 
satisfies the same quantum Liouville equation if the forward time t is used (the actual 
time is r — t in the same sense as in the classical case) . Here is the anti-linear time- 
reversal operator and a fundamental symmetry of the Hamiltonian QH{t) = H{t)Q is 
assumed |45]. Noting that QpQ~^ = —p, the commutation of H and is, in the present 
context, equivalent to the assumption that the classical Hamiltonian is symmetric with 
respect to the inversion of momenta. Then, we can prove that Tr(plnp) is also time 
invariant. In general, both von Neumann entropy and the quantum relative entropy 
are invariant under any unitary transformation [28]. Hence, each term in the quantum 
version of the relative entropy [26] : 

^Q(pWI|p(t)) = Tr(p(t)lnp(t))-Tr(p(t)lnp(t)), (36) 

can again be shifted to the initial and final time, respectively: 

Dq(pIIp) =Tr[p(0)lnp(0)]-Tr[p(r)lnp(r)], (37) 

which is analogous to the classical result ( ITOl) . 

By using the appropriate choices of initial and final equilibrium density matrices, 
one can easily derive the quantum versions of all the aforementioned classical results. 
As an illustration, we present the quantum version of Eq. ([1]). Substituting canonical 
equilibrium density operators 
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where the partition function is Z(T, A) = Tr e into Eq. fl37l) . we again obtains 

kTDM\p) = (W) -AF= (ly)diss . (39) 

5. Measuring the relative entropy 

While our general formula for entropy production is exact and may be useful as the 
starting point for other theoretical investigations, the experimental or even numerical 
measurement of the phase space density may practically not be feasible. As shown 
in Ref. [10], one can still have a good estimate of the relative entropy through the 
measurement of a limited number of physical quantities. With this partial information, 
it is clear that the distinction between the forward and backward processes becomes 
harder. Mathematically, this is tantamount to saying that the relative entropy decreases 
when only partial information of the system is used: 

AS = kD{p\\p) > kD{p\\p) . (40) 

where p and p are the probability distributions of the quantities used to describe the 
system in the forward and backward process, respectively. We have checked the accuracy 
of this inequality in different situations fiUl H3l Elj. It is remarkable that, in the 
canonical scenario with T = T', the variables coupled with X{t) are enough to saturate 
the inequahty [21]. 

The quantum case is more involved. One can lose information about the state of 
the system in different ways: by performing specific measurements or by tracing out the 
degrees of freedom of a subsystem, such as a thermal bath. In any of these two cases, 
we can derive an inequality similar to fj40|) . 

Consider first a measurement of the observable = J2i ^{Pi where the projection 
operator satisfies the closure relation: X)? ^« = I ■ The probability to obtain uoi is given 
by Pi = Tr(pPj) and pi = Tr(pPj) for the forward and backward processes, respectively. 
Then, from the joint convexity of the relative entropy [28], we have 

AS = kDM\p) > kD{p\\p). (41) 

where the discrete version of classical relative entropy is defined as 

D{p\\p)=J2p^^rl^■ (42) 

i Pi 

It should be stressed that the corresponding von Neumann entropy shows the 
opposite inequality, namely, 

S{p)>SM- (43) 
Since some information is lost, the Shannon information entropy increases. However, 
this result should not be interpreted as the increase of thermodynamic entropy due 
to the quantum measurement since the quantum states after measurement are not in 
thermal equilibrium and the Shannon entropy does not correspond to thermodynamic 
entropy. The inequality merely indicates that an estimate of von Neumann entropy 
using the probability distribution of measured quantities provides an upper bound. 
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Next we consider compound systems. For simplicity, we consider a bipartite system 
consisting of two subsystems a and b. The total Hilbert space is the product space 
TCa+b = 'Ha ® 'Hb- If we are interested in the state of the subsystem a, the other 
subsystem can be traced out to obtain a reduced density operator pa = Tr^ Pa+b- The 
monotonicity of the relative entropy states |26j: 



Since we lose some information about the system, again we cannot find the exact amount 
of the dissipation. However, the relative entropy of reduced densities provides its lower 
bound. 

While the quantum version of the relative entropy shows the same properties as the 
classical version, it is, again, worth mentioning that the von Neumann entropy satisfies 
an unusual inequality: 



This property of sub-additivity is quite remarkable since, unlike the classical Shannon 
entropy, the entropy of a subsystem can be larger than that of the whole system. This 
observation plays an important role in quantum information theory. Furthermore, the 
role of such specific quantum features on the foundations of statistical mechanics is the 
object of an ongoing debate [iGlliTt liSlliO] . 

6. Conclusions 

We have shown that the relative entropy or Kullback-Leibler distance between the 
forward and backward state is equal to the entropy production along the forward 
processes in different scenarios, both classical and quantum. Stein's lemma [12] gives 
a precise meaning to the relative entropy D{pa\\pb) between two distributions pa and 
Pb'- ii n data from ps are given, the probability of guessing incorrectly that the data 
come from pA is bounded by 2~"-^*^^'*ll'''^\ for n large. Therefore, the relative entropy 
is a quantitative measure of the distinguishibility between the two distributions. In 
the case of the forward p{t) and backward p{t) states of a system, the relative entropy 
D{p{t) I \p{t)) measures the distinguishibility of the direction of time. Therefore, our main 
result AS = kD{p(t)\\p(t)), is a quantitative relationship between entropy production 
and the arrow of time. 

We have proven this identity in a number of scenarios, by choosing the appropriate 
initial conditions for the forward and the backward process. The identity is valid 
whenever p(0) and p(r) are given by equilibrium states or factorized equilibrium states 
(such as in the multicanonical case). 

From a fundamental point of view, one can of course argue, as we already mentioned 
in the introduction, that we have merely shifted the issues of relaxation and entropy 
production to the purported properties of the bath. While this is, strictly speaking, 
indeed the case, the resulting expression of the entropy production in terms of relative 
entropy, Eq. ([5]), incorporates two fundamental properties, namely time-translational 



AS = kDQ{pa+b\\pa+b) > kDqip 

a\\Pa)- 



(44) 



SniPa) - Sn{pb)\ < S^{pa+b) < St^{pa) + SniPb)- 



(45) 
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invariance (Liouville equation) and time-reversibility of the laws of physics. Hence, 
our explicit expression for the entropy production quantified as the statistical time 
asymmetry, is very much in the spirit of Onsager's work on the symmetry of the 
Onsager coefficients, resulting from the same ingredients, Liouville's theorem and micro- 
reversibility. 

Finally, in the quantum case, while the extension of our results to quantum processes 
looks straightforward, at least mathematically, there are intriguing fundamental 
questions. For example, how do decoherence, quantum measurement and quantum 
entanglement contribute to the fluctuation theorems and ultimately to the second 
law? On the other hand, the von Neumann entropy and the quantum relative 
entropy can detect such processes through the change in the density operators. In 
fact, these entropies are used to quantify the degree of entanglement in quantum 
information theory [JHlEZj- Therefore, our novel approach for irreversibility in quantum 
processes allows one to investigate quantum non-equilibrium thermodynamics involving 
entanglement and decoherence from a novel perspective. 
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